Skip to content

UX: add progress indicators for translation tasks#1257

Merged
jmartin-tech merged 3 commits intoNVIDIA:mainfrom
jmartin-tech:feature/multilingual-add-progress
Jun 27, 2025
Merged

UX: add progress indicators for translation tasks#1257
jmartin-tech merged 3 commits intoNVIDIA:mainfrom
jmartin-tech:feature/multilingual-add-progress

Conversation

@jmartin-tech
Copy link
Collaborator

When a target_lang is specified some preparation prompts and triggers may batch work without providing any indicator to the console. To better communicate to the user the status of activity by the process provide a progress bar when processing these items in batch.

Verification

List the steps needed to make sure this thing works

  • configure with a target_lang other than "en" valid for your env: i.e. hf_RigoChat_gpu_riva.yml:
run:
  target_lang: "es"
  langproviders:
    - language: es,en
      model_type: remote
      uri: 0.0.0.0:50051
      use_ssl: false
    - language: en,es
      model_type: remote
      uri: 0.0.0.0:50051
      use_ssl: false
plugins:
  probe_spec: continuation,goodside,latentinjection,leakreplay,lmrc
  generators:
    huggingface:
      Model:
        name: IIC/RigoChat-7b-v2
        hf_args:
          device: cuda
          trust_remote_code: true
          torch_dtype: float16
  • RIVA_API_KEY=fake python -m garak --config hf_RigoChat_gpu_riva.yml -m huggingface -n IIC/RigoChat-7b-v2 -g 1
  • Verify progress bars are displayed for long running translation tasks for triggers and prompts

Copy link
Collaborator

@leondz leondz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty elegant

one refactoring comment

preparation_bar = tqdm.tqdm(
total=len(self.triggers),
leave=False,
colour=f"#{garak.resources.theme.PROBE_RGB}",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

considering different colour

self.langprovider = self._get_langprovider()
if self.langprovider is not None and hasattr(self, "triggers"):
# check for triggers that are not type str|list or just call translate_triggers
preparation_bar = tqdm.tqdm(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for ease of maintenance i'd like a central location for these bars, maybe within langproviders or langservice, so we can update them all in one go. should probably be invoked from the current calling locations. one change i'd like is the name of the langprovider being used, to be in the description, for consistency - this seems easier to do w/ a centralised def. a tqdm object might do the trick.

Copy link
Collaborator Author

@jmartin-tech jmartin-tech Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are tqdm objects already.

I disagree with having the langprovider's name such as langprovider.remote.RivaTranslator in the description here, how does that improve the user's context on progress of the run? I could see some value in inserting the probe name since that is not displayed yet when this progress bar starts.

These should eventually be managed via a display or console presentation layer, I suggest we defer consolidation until we can build out a more consistent pattern.

The callback pattern used distinctly decouples the action taken from the notification that the service is triggering to make this easier to refactor later.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, correct me if I'm wrong, progress bars in garak are typically labelled with a description of the class that's running and maybe some salient config too.

If we put in the probe name, do we give the experience of seeing a bar with a probe name complete while language provisioned, and then another bar with similar description start from zero again while prompts are sent to the generator?

suggest we defer consolidation until we can build out a more consistent pattern.

The pattern in this PR is pretty consistent, but repeated - can you say more about the expected level of consistency?

Copy link
Collaborator Author

@jmartin-tech jmartin-tech Jun 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you say more about the expected level of consistency?

Consolidation of all the scattered tqdm usage into a display managing layer. The repeated pattern here is just two method calls, one to instantiate the bar and one to close it.

Existing progress bars often refer to the plugin that creates them or caused them to be created, in this case I can see mimic of how a tqdm for a buff that is mutating prompts is labeled:

desc=f"📥 Buffing probe: {probename}/{self.fullname}"

Translation is a very similar action so I could see expanding the specificity of this tqdm to be:

desc=f"Preparing triggers: {self.probename.replace("garak.", "")}"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are seven very similar tqdm.tqdm calls in this PR all covering language provision. My question is, ist this the time to establish a pattern for encapsulating these, given that style & descriptions are likely to be very similar also? Doesn't need to block, current code just looks a bit far from DRY

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all one method call that would need similar params passed to a new method, in my opinion this does to not make sense to refactor for this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aight. It does feel marginal - almost all the params are things that one would specify to an encapsulation anyway. Will send descr & RGB const commits

@leondz leondz self-assigned this Jun 26, 2025
@leondz leondz added cli Command-line interface functions ux Interface & interaction improvements labels Jun 26, 2025
@leondz leondz self-requested a review June 27, 2025 13:34
Copy link
Collaborator

@leondz leondz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jmartin-tech jmartin-tech merged commit 023225e into NVIDIA:main Jun 27, 2025
11 of 13 checks passed
@jmartin-tech jmartin-tech deleted the feature/multilingual-add-progress branch June 27, 2025 16:42
@github-actions github-actions bot locked and limited conversation to collaborators Jun 27, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cli Command-line interface functions ux Interface & interaction improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants